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Abstract 

We study the problem of determination of the sign of Amg^ , or the neutrino mass hi- 
erarchy, through observations of atmospheric neutrinos in future detectors. We consider 
two proposed detector types : (a) Megaton sized water Cerenkov detectors, which can 
measure the event rates of v^j. + Vfj, and v^ + ^e and (b) 100 kton sized magnetized iron de- 
tectors, which can measure the event rates of and v^. For energies and path- lengths 
relevant to atmospheric neutrinos, these rates obtain significant matter contributions 
from P^e, P^^ and Pee, leading to an appreciable sensitivity to the hierarchy. We do a 
binned \^ analysis of simulated data in these two types of detectors which includes the 
effect of smearing in neutrino energy and direction and incorporates detector efficiencies 
and relevant statistical, theoretical and systematic errors. We also marginalize the 
over the allowed ranges of neutrino parameters in order to accurately account for their 
uncertainties. Finally, we compare the performance of both types of detectors vis a vis 
the hierarchy determination. 
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1 Introduction 

Our understanding of neutrino physics has progressed impressively over the last decade owing 
to outstanding results from solar, atmospheric, reactor and accelerator based neutrino oscil- 
lation experiments. For three neutrino flavors, the picture of neutrino masses and mixings 
emerging from these results is: 

• There are two independent mass squared differences given by jAnig^l ~ 10"'^ eV^ and 
Amli ~ 10-5 eV^ ; 

• Of the three mixing angles, two (^12 and 623) are large, while the third mixing angle 
(^13) is small and constrained only by an upper bound. 

In Table [H we summarize the present bounds (the best-fit values and 3a ranges) of these 
parameters. 



Parameter 


Best-fit value 


3cr range 


Am^JlO-W] 
|Amii|[10-3eV2] 

sin^ 9i2 

sin^ 6^23 

sin^ 6^13 


7.9 
2.5 
0.3 
0.5 
0.0 


7.1 - 8.9 
1.9 - 3.2 
0.24 - 0.40 
0.34 - 0.68 
< 0.041 



Table 1 : Best- fit values and 3a allowed intervals for three-flavour neutrino oscillation parameters from com- 
bined analysis of global data including solar, atmospheric, reactor (KamLAND and CHOOZ) and accelerator 
(K2K and MINOS) experiments [1]. 

These results have also delineated the main goals of neutrino oscillation research over the 
next decade, which can be summarized as follows : 

• Improved precision on the mass squared differences (Am^j^ and Amg^) and the mixing 
angles (6'23 and 6*12) ; 

• Determination of 6*13, especially ascertaining if its value is non-zero ; 

• Determination of sign(Am3]^) or the hierarchy of neutrino masses ; 

• The presence, or otherwise, absence of CP violation in the leptonic sector and the value 
of the phase 6cp- 

^We use the convention Amj? = m? — mf. 
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The experimental realization of the above goals is a complex task. Hence, a large number 
of neutrino experiments are being constructed or being planned to work towards achieving 
these aims. 

Our focus in this work is to bring out the potential of megaton water Cerenkov detectors 
{e.g. HK [2,3], UNO [4] or MEMPHYS [5]) and magnetized iron detectors {e.g. INO [6]) vis a 
vis the goal of determining the neutrino mass hierarchy. Both of these detector-types plan 
to use atmospheric neutrinos as one of their primary sources. Each category of detector has 
a special ability, in addition to large statistics resulting from its massive size. In the case of 
water Cerenkov detectors, one can detect both muons and electrons, whereas in the case of 
magnetized iron detectors, it is possible to study muons and anti-muons separately by charge 
discrimination. These advantages, together with their size, and the wide ranges in energy and 
path-length made available by atmospheric neutrinos, make these detectors powerful tools for 
studying sign(Am3^). 

Solar neutrino data require Amg^ to be positive. However, data from atmospheric neutrino 
experiments as well as accelerator neutrino experiments (K2K and MINOS) constrain only the 
magnitude of Am^^^ but not its sign. If sign(Am3]^) > 0, then we have the mass pattern, 
m3 ^ m2 ^ mi, which is similar to that of the charged leptons. This is called the normal 
hierarchy (NH). If sign(Am3]^) < 0, then the mass pattern is m2 > nii ^ m3. This is called the 
inverted hierarchy (IH). These statements, of course, are meaningful provided a discernible 
and non-trivial hierarchy exists among the neutrino mass eigenstates, or, in other words, 
the lightest neutrino state is almost massless. If future experiments on tritium beta decay or 
neutrinoless double beta decay show that the absolute scale of neutrino masses is, for instance, 
^ 0.3 eV, then all three neutrino masses must be nearly degenerate. However, one may still 
ask the question if sign(Am3^) is positive or negative. 

Attempts to construct unified theories beyond the Standard Model depend crucially on 
the hierarchy. In fact, one way to classify families of unification models is via the hierarchy 
they assume as input. It is thus an important ingredient in our quest for a unified theory, 
and its determination would in essence eliminate or strongly disfavour large classes of such 
theories and considerably narrow the focus of this search. 

Generally speaking, determination of the mass ordering in oscillation experiments requires 
the observation of measurably large matter effects {i.e. long baselines) and a not too small 
value for sin^ 2^13 ( ^ 0.05). This limits the sensitivity of presently planned superbeam ex- 
periments to hierarchy determination as the baselines involved are moderate ( ^ 800 km) and 
significant matter effects do not develop. Also at these baselines, for the P^e channel which 
these experiments use, the oscillation probability depends jointly on three unknown parame- 
ters: sign(Am3j^), the phase, 6cp and the mixing angle, ^13. This results in a (^cp? sign(Am3j^)) 
degeneracy where acceptable solutions corresponding to the two signs of Am|i can appear 
at different values of 6cp and ^13 [7] @. To overcome this problem, the synergistic use of two 
experiments [13-18] or two detectors at different baselines [19,20] has been suggested. Among 
the combinations studied, the planned superbeam experiments T2K [21, 22] and the NuMI 
Off-axis experiment NOz/A [23] may be able to infer the neutrino mass hierarchy [24-27]. 

^The Sep dependence tends to decrease for longer baselines, and in fact vanishes at (and around) the so 
called "magic baseline" 7000 km) [8-12]. reappearing subsequently at longer baselines. 
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In this paper, in order to ascertain the neutrino mass hierarchy, we focus on the appreciable 
resonant matter effects encountered by atmospheric neutrinos with energies between 2—10 
GeV which traverse distances between 4000—12500 km inside the earth. Until very long 
baseline experiments using /5-beams or neutrino factories are built atmospheric neutrinos are 
the only source that permit us to exploit these spectacular effects [11,28-33]. Moreover, for 
these energy and pathlength ranges, it turns out that the effects of 6cp degeneracy are not very 
significant [34]. Therefore, lately, the subject of hierarchy determination using atmospheric 
neutrinos has received considerable attention [34-39]. Recent work has also emphasized the 
degeneracy resolving power of atmospheric neutrino data [40-42] in the context of water 
Cerenkov and magnetized iron detectors respectively. In particular, the issue of the resolution 
of {623,11/2 — 623) degeneracy in magnetized iron detectors has been considered in [43,44]. 

In our earlier work on this subject [34,45] we had used the total event count rates (in the 
energy range 2—10 GeV and pathlength range 4000—10000 km) to examine the determination 
of the neutrino mass hierarchy for iron magnetized calorimeters and water Cerenkov detectors 
respectively. Since we used total event rates, the effects of energy and angular smearing were 
not very important in those analyses. We had also made the simplistic assumption that the 
values of the neutrino parameters are known exactly. In this paper, we relax this assumption 
and take into account the uncertainties in the determination of neutrino oscillation parameters. 
Under these circumstances, the total event rates can no longer give good sensitivity and 
therefore one has to do a binned spectrum analysis including appropriate angular and energy 
smearing. In what follows, we do a detailed study of the effect of these refinements on the 
hierarchy sensitivity for both water Cerenkov and iron calorimeter detectors. 

The hierarchy sensitivity of iron calorimeter detectors was studied in [38] under the ap- 
proximation that Am^^ = 0. In our numerical work we take Am^^^ to be non-zero because, 
in principle, it becomes important in the limit of small 613. However, the marginalization 
over |Am|]^| washes out the hierarchy sensitivity due to non-zero Am2]^ in this limit. Allowing 
a non-zero value for Ani^i allows us to study the variation of with 6cp- We also com- 
pare and contrast the performance of a magnetized iron calorimeter detector with a water 
Cerenkov detector in discriminating between the two hierarchies. 

In the next section, we discuss and summarize the matter sensitivities of various oscillation 
probabilities that are relevant to the atmospheric neutrino signal and give analytic expressions 
for maximum hierarchy sensitivity. Section 3 describes the prototype detectors we have used 
and the details of the statistical analysis on which our results are based. We also include a 
description of our treatment of energy and angular smearing and our choices of the number of 
bins for each of these variables. Section 4 is devoted to a description of our numerical procedure 
on the sensitivity to the hierarchy. Section 5 summarizes our results and conclusions. 

2 Earth-matter effects in atmospheric neutrino oscillation proba- 
bilities 

In calculating atmospheric electron and muon event rates, the neutrino oscillation probabilities 
corresponding to the disappearance channels P^^ and Pee and appearance channels P^e and 
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Figure 1: The probabilities Pf^cPtin and Pee in matter plotted versus neutrino energy for two baselines, 
3000 km (left panel) and 5000 km (right panel). The solid curves correspond to NH (Am|]^ > 0), while the 
dashed curves are for IH (Am§j < 0). Here, we use the neutrino parameter values listed in Eq. [T] 

Pe^ are of direct relevance For the long path-lengths under consideration here, we need 
to explicitly take into account the varying density profile of the earth, and for this purpose 
we use the Preliminary Reference Earth Model (PREM) [46] . In Figures [T] and El we plot 
the three probabilities P^e, P/ifi and Pee as a function of energy for both normal and inverted 
hierarchies for four different path- lengths in the range 3000—10000 km. These curves result 
from numerically solving the full three flavour neutrino propagation equation through the 
earth. In obtaining them we have used the following values for neutrino parameters: 

\Aml^\ = 2.5 X 10"^ eV^ , Am^^ = 8.0 x 10"^ eV^ 

sin^^iz = 0.31 , sin^ 2^23 = 1-0 , sin2 2^i3 = 0.1 (1) 

Figures [T] and [2] demonstrate the following qualitative features : 
■^Pq^ denotes the probability for transition from VfS- 
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Figure 2: The probabilities P^c,P/^/j and Poo in matter plotted versus neutrino energy for two baselines, 
7000 km (left panel) and 10000 km (right panel). The solid curves correspond to NH (Am|]^ > 0), while the 
dashed curves are for IH (Am§]^ < 0). Here, we use the neutrino parameter values listed in Eq. [T] 
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• The differences between the probabihty values for the two types of hierarchy in all the 
channels are maximized in the energy range 4—10 GeV. 

• Matter effects in P^e in case of a NH manifest themselves by a rise over the corresponding 
value for an IH over the range of energies and baselines considered in these figures. 

• On the other hand, matter effects in P^^ for a NH manifest themselves as a decrease over 
the IH value in certain energy and baseline ranges and an increase in others {e.g. for 
baselines around 10000 km). 

• The value of Pee for a NH consistently shows a fall compared to the value with an IH 
for all four baselines in the energy range 2—10 GeV. This fall can be as high as 100% 
{e.g. for baselines around 10000 km). 

We discuss these features below, using the one mass scale dominant (OMSD) approxima- 
tion, which amounts to neglecting the smaller mass-squared difference Am^^ in comparison to 
Amg^. The condition for this approximation to be valid is 

« 1 (2) 

Alternatively, this corresponds to L/E << 10"^ km/GeV. For multi-GeV neutrinos, this con- 
dition is violated only for a small fraction of events with E ~ 1 GeV and L > 10^ km. Hence, 
effectively Eq. [2] is valid for most of the energies and pathlengths under consideration here. 
Additionally, the OMSD approximation is not valid for very small ^13, since the terms con- 
taining the small parameter Ani^i can be dropped only if they are small compared to the 
leading order terms containing 6*13 0. 

Making this approximation and assuming constant matter density, the probabilities P^e, 
Pee and P^^ in matter are given by 



fie 



sm 



13 



sm 



1.27 (Am; 



31) 



L 
E 



(3) 



The probability for the time-reversed transition P™ is same as P™ with the replacement 
^cp ~^ ~^cp for a symmetric matter density profile like the PREM profile. The OMSD 
analytical expressions are insensitive to 6cp, hence, P™ = P™ in this limit. In general, the 
two probabilities will be equal only when 6cp = 0. 



P^e = 1 - sin' 26"^^ sin' 



1.27 (Am^,)- 



(4) 



Expressions of probabilities eontaining corrections of the order of Am2i/Am|]^ can for instance be found 
in [47-50]. 
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fj.fl 



1 - cos^ ^5°3 sin^ 2^23 sin^ 



sin^ ^5^3 sin^ 2^23 sin^ 



sin^ sin^ 2^^3 sin^ 



1.27 



Am|i + A + (Am3i 



2 L' 
E 



1.27 {AmiX ^ 



\_27 |^ Am^,+A-(Am^,r ^| L" 



(5) 



The mass squared difference (Amg^)™ and mixing angle sin^ 2^5^ in matter are related to 
their vacuum values by 



2 Mn 



(Am^,) 



(Ami^ cos 2^13 - A)2 + (Am|i sin 2^13)^ 



sin 29 



Am^i sin 29 



13 



13 



v/(Am2^ cos 2^13 - A)2 + (Am|i sin2^i3)2 



(6) 



where 



A = 2v/2GFncE = 2x 0.76 x 10"^ x Ye 



E 



GeV 



eV^ 



is the MSW matter potential [51,52] which depends on the Fermi coupling constant, Gp, the 
number density of electrons, Ue and energy of the neutrinos, E. Ye is the fraction of electrons, 
which is ~ 0.5 for earth matter and p is matter density inside earth. The anti-neutrino 
probabilities can be written down by making the replacement A ^ -A in above equations. 

We note that (Am^^)™ and sin 26*5^ can assume very different values for a NH and an IH, 
leading to hierarchy dependant differences in all three probabilities given above. Furthermore, 
large changes in the values of these probabilities arise not just due to the resonance occurring 
when sin^ 26*5^ — > 1, z.e. A = Am|]^ cos26'i3, but also due to the matter dependence of the 
oscillatory L/E terms contained in them. Thus the full effect results from the product of the 
two types of terms assuming values significantly different from those in vacuum [29,34,53]. 

The matter resonance, which occurs for neutrinos when the hierarchy is normal and for 
anti-neutrinos when it is inverted, requires 



E(GeV) ~ E,es(GeV) 



1 



2 X 0.76 X lO-^Yp 



eV2 



(7) 



If we use an average density calculated using PREM profile of earth, then we get the resonant 
energy at various baselines, as listed, in Table [21 

Clearly, hierarchy sensitivity is enhanced when the difference 



AP 



a/3 



8 



L (km) 


Pavg(g/Cc) 


Eres (GeV) 


3000 


3.32 


9.4 


5000 


3.59 


8.7 


7000 


4.15 


7.5 


10000 


4.76 


6.6 



Table 2: Values of Eics at 13 resonance in case of OMSD are listed as a function of baseline and the average 
density pavg- We have used |Ani§i| = 2.5 x 10~^ eV^ and sin^ 26*13 = 0.1 in Eq. [71 



is large 0. It is useful to use the OMSD expressions given above to examine the conditions 
required for this to occur, and to co-relate them with the large differences between the NH 
and IH curves visible in Figures [T] and [2] (which, as mentioned earlier, have been obtained 
using the full three flavour evolution equation and the PREM profile). We proceed to do this 
below for each of the three probabilities in turn. 

2.1 P;.e 

From Eq. [HI we can see that this probability reaches its maximum value of sin^ ^23 when the 
resonance condition, sin^ 26'^^ = 1 and the condition sin^(1.27(Am3^)™L/E) = 1 are simulta- 
neously satisfied. The second condition yields an energy (henceforth referred to as the matter 
peak energy) given by 



1.27 

(2p + 1) 7r/2 



Epeak = T^^T-^V^ i^^lX L ; p = 0, 1, 2, . . . (9) 



Thus, the condition for obtaining a maxima in P^c can be expressed as Eres = E™eak- This 
determines the distance for maximum matter effect via 

[pmT"^ = 5.18 X 10^ vr i^Pi-^ km g/cc (10) 

tanzt7i3 

The above equation is independent of Amgj^ but depends on the vacuum value of Oi^. Note, 
from the above equation however, that for a given resonant energy, whether or not the oscil- 
latory term in L/E can become maximal in a meaningful way depends on: 

(i) p being low enough and sin^ 2^13 large enough that the resultant L is less than the earth's 
diameter, = 12742 km. For sin^ 2^13 = 0.1, the only allowed value of p is p=0, and 
this gives L=10200 km in conjunction with pavg = 4.8 g/cc. 

(ii) sin^ 26'i3 not being so small that the OMSD approximation, which we use for the discus- 
sion in this section, is invalidated. 



^Here a, (3 may be e or fx. PJ^^(IH) is computed in each case by reversing the sign of Amg^ in the expression 
for P- . 
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For our discussion which follows below leading to Eq. [TT] and Eq. [T51 the first restriction on 
sin 2^13 limits sin2 2^i3 ^ 0.04 0. 

Under these conditions, for a particular value of 6'i3, one may write, 

sin^ 2^1 



A pmax 



sin^ e- 



23 



'13 



4-3 sin^ 2^13 



sm 



(2p 



TT 



1 



2 sin 2^ 



\/4 - 3 sin^ 29 



13 



13 
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Numerically, this reduces to AP™'^^ ~ 0.5 for ^23 = 7r/4 and sin^2^i3 = 0.1. Comparing 
with the top panel of the right-hand set of plots in Figure [21 we see that both the baseline 
value and AP™^^ are in very good agreement with these estimates. For smaller baselines 
although the resonance condition is achieved the oscillatory term remains < 1 and thus the 
rise in P™ in matter for NH is lower. However, since this is an appearance channel even with 
a moderate increase in probability it is possible to do interesting physics if the backgrounds 
are well understood. 



2.2 P, 



From Eq. H] it is clear that the condition for obtaining maximum matter effect in P™ is the 

= Epeak- This gives 



same as that for P™, i.e., E 



A pmax 
ee 



sin^ 26 



13 



4-3 sin^ 2^13 



sm 



(2p 



1 



2 sin 2^ 



^4-3 sin^ 26 



13 



13 



(12) 



This quantity is maximized when P™ (NH) is at its minimum value of and P™ (IH) is at 
its maximum value, which is close to 1. For sin^ 2^13 = 0.1. and the earlier baseline value of 
~ 10000 km, this reduces numerically to ^ —0.98, which is manifest in the bottom panel of 
the right-hand set of curves in Figure [2] [54] . 



2.3 P 



The muon disappearance probability is a somewhat more complicated function as can be seen 
from Eq. [5l In order to understand where and why maximal hierarchy effects arise, we first 
note from Figures [H (central right-hand panel) and [2] (central panels in both left-hand and 
right-hand sets) that large hierarchy sensitivity occurs when the neutrino energy is close to 
Eres (cf Table 2) and in the vicinity of a peak or dip in P™^(IH). Since the matter effect 
in neutrinos is negligible for IH, these curves closely follow the vacuum probability, and in 
particular have peaks and dips at the same locations. 



Noting that the vacuum peak of V^n occurs when 



1.27 



e: 



p TT 



(13) 



peak 



^ sin^ 2013 ^ 0-04 is obtained by substituting L = Ld = 12742 km and p ~ 7 g/cc. This value is a 
conservative estimate considering the fact that density at the center of earth is not very weU-known and that 
at such large distances, the constant density approximation is not very good. 
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and, setting E'^^^y. = Ercs, we obtain 

[pmT^ - PTT X 10^ X cos 2^13 km g/cc 



(14) 



For p=l, i.e., when the resonance is near the first vacuum peak this leads to 7000 km 
(taking the average density to be 4.15 g/cc), which is borne out by the middle panel of the 
left column of Figure [2l The magnitude of this fall can be estimated by 



A pmax,pcak 



sm 



TT 



p - sin 26^13 



sm 



p TT sin 26 



13 



sm 



p| V4-3sin2 2ei3 



+ 



1 sin^ 2^13 
4 4-3 sin^ 26 



sm 



13 



pvr V4-3sin^2^i3 



(15) 



-0.4, which is borne out 



Numerically, this gives a drop in P™^ (NH) relative to P™^ (IH) of 
by central panel in Figure [2] near the region of peak in IH curve. 

The right column of Figure [2] also shows that at 10000 km there is a rise in P^^(NH) 
relative to P^^(IH) near a dip in the IH probability at ~ 6 GeV. The condition for dip in P^^ 
in vacuum is 



Am^ L 

1.27 



E 



(2p 



dip 



71 
2 



(16) 



This gives the position of the second vacuum dip {i.e. when p = 1) for L = 10000 km to be 
around ~ 6.7 GeV. Table [2] shows that at 10000 km the resonance energy is ~ 6.6 GeV. Thus, 
we have the condition Eres ~ E^jp satisfied here for p=l. Using this, one can similarly estimate 
the magnitude of the corresponding rise, and find it to be in good agreement with the figure, 
i.e. about 22% for p = 1 0. 

It is interesting to note from Eq. [TT]and from Figure [2] that at this baseline there is a 50% 
rise in P^c probability over IH. Since AP^,- = — (AP^e + ^P^j/i) this implies a 70% matter 
induced decrease in the P^r probability, which is a transition probability between two species 
of neutrinos for which matter effect occurs only via neutral current interactions and is identical 
to both. This dramatic matter driven decrease is a genuine three flavour effect and was first 
pointed out in [55]. 

To summarize, we have discussed the important features of 3 generation oscillation prob- 
abilities in matter corresponding to appearance and disappearance channels. In order to do 
that, we have (in this section only) worked in the OMSD approximation, which allows the use 
of tractable analytic expressions. We have emphasized those features which percolate into the 
disappearance probability for electron and muon events and modify the expected atmospheric 



^In order to simplify this expression, we have set cos^ 20i3 — 1. 

^For p = 0, the value of AP^^ is small since matter effects are not appreciable at the associated baseline 
of 5000 km. 
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neutrino signal in water Cerenkov and magnetized iron detectors in a hierarchy dependent 
manner, and examined the cases where these effects are maximized. However, the full effects 
are spread over a wide band of energies and baselines (4—12 GeV and 3000—12000 km|^ re- 
spectively). Inspite of the fact that the effects vary in significance over these ranges for each 
of the probabilities, cumulatively they provide a powerful discriminator of the hierarchy. We 
exploit this fact by doing a bin-by-bin analysis of simulated data to determine the potential 
for hierarchy determination. 

3 Detector Characteristics 

In this section we describe the details of the two types of detectors that we have considered 
for our study. 

3.1 Water Cerenkov Detectors 

We consider a prototype megaton Water Cerenkov detector based on the proposed Hyper- 
Kamiokande (HK) detector. HK will essentially be a scaled up version of the Super-Kamiokande 
(SK) detector with the total volume increased to 1 Mt and a fiducial volume of 545 Kt. R&D 
initiatives are in progress currently to study the non-trivial physics and engineering issues 
which arise due to this scaling up in size [59]. 

Since water Cerenkov detectors can discriminate between muon and electron events, the 
full atmospheric neutrino spectrum can be studied. They are, however, insensitive to lepton 
charge, thus neutrino and anti-neutrino events must be added together. In the following, we 
add the events of u^, whose charged current (CC) interactions produce a fi~ , and those of z/^, 
whose CC interactions produce a /x"*", and call the sum to be muon events. Similarly, we add 
the CC events of Uc and and call the sum to be electron events. 

In this case the hierarchy sensitivity is determined by the difference in the total number 
of events. For instance for muon events this quantity is 

AN = (n;:« + n;:») - (Nji + Nf,) = (n;:_^ - N;f_) + (n;:,^ - n;« ) = an,. + an,, (it) 

If the hierarchy is normal, matter effects induce large changes in neutrino appearance and 
disappearance probabilities and hence in N^?, while leaving anti- neutrino probabilities and 
hence N|^^ essentially the same as the vacuum probabilities. If the hierarchy is inverted, then 
anti-neutrino probabilities undergo large changes and neutrino probabilities remain the same. 
Thus AN,- and AN,+ have opposite signs leading to cancellations. But AN,- is larger by a 
factor 2.5 — 3 because the neutrino-nucleon cross sections are higher by this factor and hence 
the cancellation is only partial. A similar reasoning holds for electron events. This leads to 
hierarchy sensitivity in water Cerenkov detectors. It is not as good as that of detectors with 

^Beyond 10500 km, the neutrinos start traversing the core, causing the onset of mantle-core interference 
effects [36,56-58]. Our fuU numerical calculations incorporate the difference between NH and IH probabilities 
due to these effects. 
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charge discrimination capability but the proposed megaton mass overrides this disadvantage 
and provides the statistics necessary for a determination of the hierarchy. 

Since no simulation studies for atmospheric neutrinos are available for HK we assume the 
same detector characteristics as in the SK detector but with increased statistics. Our results 
are for an exposure of 1.8 Mt yr, which corresponds to 3.3 years of running time. In our cal- 
culation, we have put in a lepton energy threshold of 1 GeV, since Am|^ driven matter effects 
arise in this energy range and the determination of sign(Am3j^) is better achieved with higher 
energy neutrinos. This leads to a threshold correction in the cross section [38], which we in- 
corporate into our calculations. We use the detection efficiencies of the SK experiment [60,61] 
for multi-GeV one-ring muon (both fully and partially contained /^-like) events and electron 
(e-like) events. The L/E analysis of SK muon-data has demonstrated the feasibility of re- 
constructing the neutrino energy in a water Cerenkov detector from Monte Carlo simulation 
by fitting it to the total energy of the charged particles [61]. Similarly the direction of the 
neutrino can also be determined from the reconstructed direction of the muon [62]. For fully 
contained (FC) (partially contained (PC)) multi-GeV muon events the energy smearing is 17% 
(24%) while the angular smearing is 17° (10°). In our calculations for purposes of comparison 
with magnetized iron calorimeter detectors, we use an overall 15% energy smearing and 10° 
angular smearing for both types of events. While this is somewhat optimistic, we will discuss 
in detail the extent to which the sensitivity to hierarchy depends on smearing. 

Both muon and electron events in this detector have contributions coming from back- 
ground processes. The backgrounds in the FC events are due to cosmic ray muons, PMT 
flashes and neutron events from the rock surrounding the detector [63]. For PC events the 
cosmic ray muons constitute the main background. These backgrounds can, however, largely 
be eliminated during the data reduction procedure [63]. The remaining backgrounds occur 
due to (a) neutral current (NC) events and (b) (uq) induced CC events for electron (muon) 
data sample. The expected backgrounds in the case of atmospheric neutrino interactions are 
estimated by the SK Monte Carlo and are given in [63]. For multi-GeV one-ring muon events 
the contamination due to above processes is estimated to be about 0.3% whereas for multi- 
GeV one-ring electron events have a background of about 10% from NC events and about 7% 
from z/^ induced CC events. 

3.2 Magnetized Iron Detectors 

We consider a prototype magnetized iron detector along the lines of the India-based Neutrino 
Observatory [6] (INO). This detector is expected to have a modular structure with an initial 
running mass of 50 kT, building up to a final mass of 100 kT. It will consist of 140 layers 
of iron plates about 6 cm thick, with gaps of about 2.5 cm between them housing the active 
elements. These have been chosen to be RPC's (Resistive Plate Chambers), made of glass 
or bakelite and containing a mixture of gases. The (50 kT) structure is divided into three 
modules, with an overall lateral size of 48 x 16 meters and a height of 12 meters. A magnetic 
field of about 1.3 Tesla will provide charge discrimination capability to the detector. We 
consider an exposure of 1.0 Mt yr , which corresponds to 10 years of running time. 

The high density of iron renders this detector insensitive to sub-GeV muons and electrons 
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of any energy. In our calculations, we have assumed a detection threshold of 1 GeV for 
muons and included the corresponding threshold correction in the cross section. An overall 
detection efficiency of 87% and a charge identification efficiency of 100% is assumed 0. The 
magnetic field allows charge identification and thus the interactions of and Ufj_ can be studied 
separately. Thus AN^- and AN^+ defined in Eq. [T7| can each be determined. The hierarchy 
sensitivity in this case depends on the sum of the magnitudes of these quantities. Therefore 
the partial cancellation occuring for water Cerenkov detectors does not take place for charge 
discriminating detectors giving them an advantage over the former. 

The energy and the direction of the neutrino in I NO can be reconstructed from the muon 
track [6]. The energy of the neutrino is the sum total of the energy of the muon and the 
hadrons. The latter is difficult to reconstruct for individual hadrons. However, one may 
use the hit multiplicity of charged particles distinct from the muon track to calibrate the 
total energy of the hadrons in the event. It is reasonable to assume energy smearing of 15% 
and angular smearing of 10° for this detector, which are the values we adopt in our numerical 
work. As in the case of the water Cerenkov detector, we study the sensitivity of this detector's 
capability for hierarchy determination in case its resolutions are different from those assumed 
by us. 

Finally, we comment on the background signal in this detector. A preliminary study 
using a GEANT based simulation of cosmic ray muon background in I NO shows that these are 
unlikely to mimic the signal [37]. Other backgrounds can originate due to NC interactions, such 
as i^x + d (u) ^ i^x + d (u) (+ q q), where the quarks in the final state can produce mesons 
along with other hadrons. The decay of these mesons produces secondary muons which can 
contaminate the signal. However, simulations have shown that the 6 cm thickness of the 
iron plates is sufficient to absorb any pions and kaons in the 1—10 GeV range before they 
can decay. In addition, the oscillated z/t-s can produce r which can decay to muons with a 
branching ratio of 17.36% via t/,- — > /i~ + z7^ + z/t- . However the number of these secondary 
muons are expected to be small because of the higher r production threshold of 3.5 GeV. Also 
these muons are softer in energy and hence can be eliminated by suitable energy cuts [64]. 

In Table El the comparative characteristics of the two detectors HK and INO are listed. 
4 Numerical Procedure 

The total number of CC events is obtained by folding the relevant incident neutrino fiuxes 
with the appropriate disappearance and appearance probabilities, relevant CC cross sections, 
the efficiency for muon detection, the detector resolution, mass and the exposure time. The 
total CC cross section used here is the sum of quasi-elastic, single meson production and deep 
inelastic cross sections. The cross sections for the water Cerenkov detector are taken from [63] 
and for the magnetized iron detector are taken from [65-67]. For the incident atmospheric 
neutrino fiuxes we use the tables from [68] where a 3-dimensional model is employed for fiux 

^"^This charge identification efficiency is valid for neutrino energy << 1 TeV. 
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Property 


HK 


INO 


Detector Technology 


Water Cerenkov 


Iron calorimeter 


Fiducial Volume 


545 Kt 


100 Kt 


Exposure Time 


3.3 yr 


10 yr 


Energy Threshold 


1 GeV 


1 GeV 


Energy smearing 


15% 


15% 


Angular Smearing 


10° 


10° 


Detection Efficiency 


E- dependent 


87% 


Charge Discrimination 


No 


Yes 


Muon Events 


Yes 


Yes 


Electron Events 


Yes 


No 



Table 3: Properties of the two detectors considered in our analysis, HK and INO. 



calculation 0. 



For our analysis, we look at the neutrino energy range of 2—10 GeV and the cosine of the 
zenith angle (6) range of -1.0 to -0.1. These are divided into bins, and the /x~ event rate in a 
specific energy bin with width dE and the solid angle bin with width dQ is expressed as : 



d^N^ ^ 1 
dn dE 27r 



d2<l>, 



dcos^ dE 



d2<l>. 



d cos 9 dE 



Cfl 



CCC DcflF 



(1^ 



Here $^,e are the atmospheric fluxes (f^ and Uc), c"cc is the total CC cross section and Deg is 
the detector efficiency. The event rate is similar to the above expression with the fluxes, 
probabilities and cross sections replaced by those for anti-muons. Similarly, the e~ event rate 
in a specific energy and zenith angle bin is expressed as follows: 



d^Ne _ 1 

dn dE ~ 2^ 



d2<l>, 



dcos^ dE 



d2<l>. 



d cos 9 dE 



o'cc Dcff 



(19) 



with the e^ event rate being expressed in terms of anti-neutrino fluxes, probabilities and cross 
sections. 

For the HK analysis, the sum of yU~(e~) events and fi~^{e'^) events is taken to compute 
the total muon (electron) event rate, since the detector is insensitive to lepton charge. For 
the INO analysis, however, the /i~ and yU"*" event rates are separately used given its charge 
identification capability. 



"The INO facihty is expected to be housed at Pushep (lat: North 11.5°, long: East 76.6°). The Honda 
fluxes in [68] are calculated for SK (lat: North 36.4°, long: East 137.3°). However, because of unavailability 
of the fluxes at the specific INO latitude we use the Honda fluxes given at the SK site for INO . 
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4.1 Energy and angular smearing: 



We take into account tlie smearing in both energy and zenith angle, assuming a Gaussian 
form of resolution function, R. For energy, we use 



R(Et,Em) = J- exp 



(Em — Et 
2^ 



(20) 



Here, E^ and Et denote the measured and true values of energy respectively. The smearing 
width (J is a fraction of Et. Most of our calculations are performed assuming this fraction to 
be 15%. We also calculate how our results vary if this resolution fraction is reduced to 10% 
or 5%. 

The smearing function for the zenith angle is a bit more complicated because the direction 
of incident neutrino is specified by two variables: the polar angle 9^ and the azimuthal angle 
0f We denote both these angles together by fif The measured direction of the neutrino, with 
polar angle 9^ and azimuthal angle 0m, which together we denote by f2m, is expected to be 
within a cone of half angle A9 of the true direction. Since the angular smearing is to be done 
in a cone around the direction specified by (^t,0t), we cannot assume the smearing function 
to be a function of the polar angle only. If we consider a small cone whose axis is given 
by the direction 6't,0t, then the set of directions within the cone have different polar angles 
and azimuthal angles. Therefore, we need to construct a smearing function which takes into 
account the change in the azimuthal coordinates as well. Such an angular smearing function 
is given by 

(^t-^m)' + sin2^t (</>*- 0m) 



R(fit,f^m) = N exp 



(21) 



2 {MY 

Details of the computation of the normalization constant N appearing in the above equation 
are given in the Appendix \M 

The event rate with the smearing factors taken into account is given by 



d^N^ _ 1 '•loo 



dVtja dEm 27r 



' dEt y da R(Et, Em) R(a, ^^m) P^M + "^-c Pep] ^CC Deff (22) 



where we have denoted (d^$/dcos^ dE)^ = $^ etc.. Strictly speaking, the range of integration 
for the true energy of the neutrino Et should be from to oo. However, given the fact that we 
set a lower threshold of 1 GeV for the lepton energy, our choice of the lower limit of integration 
is dictated by the requirement that the neutrino be more energetic than the lepton. Taking 
an upper limit of 100 GeV for the true energy is justified because the probability of spillover 
of events from bins above 100 GeV to bins below 10 GeV due to smearing is insignificant, and 
the number of events above 100 GeV is quite small due to the steeply falling neutrino flux. 

In Figure El we show the effect of angular smearing on distribution of muon events in 
HK assuming a NH. The figure also shows the cos^m distribution for the two special cases 
of no smearing without and with oscillations. It demonstrates the washing out of oscillatory 
behaviour in the event distribution as the angular smearing width is increased. Note that 
the distribution with oscillation and without smearing shows some distortion in its shape 
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Figure 3: The muon event rate for HK (1.8 Mt yr) with different values of angular smearing and 15% energy 
smearing, as well as without smearing with and without oscillations, plotted versus cos^m taking 18 bins in 
the range -1.0 to -0.1. The energy range taken is 2 to 10 GeV, and the hierarchy is taken to be normal. 

compared to the distribution without oscillation. When the angular smearing is introduced, 
this oscillatory distortion is washed out progressively as the value of smearing is increased. For 
smearing with a large angular resolution of 15°, the distribution with oscillations resembles 
the unoscillated distribution in its shape. 

Figure H] shows the muon event distribution in HK for NH and IH for six energy bins of 
width 1 GeV each in the energy range of 4—10 GeV. From these figures it is clear that below 
the resonant energy, the difference between the normal and inverted hierarchy distributions is 
negligible (note that Efcs > 6 GeV for all baselines considered here, as described in Table H]). 
Around resonance, i.e. from the 6—7 GeV energy bin, the difference begins to be significant. 
A little above the resonance, the event numbers fall off, but a reasonable difference between 
NH and IH distributions persists, which contributes significantly to the hierarchy sensitivity. 

analysis 

We study the sensitivity to the mass hierarchy for various different values of energy 
and angular resolution. For HK, we use a binned distribution of muon as well as electron 
events in various Em and cos^m bins. For INO, we use similar binning for /i~ and /i"*" events. 

From Figure H] we note that each bin contains > 5 events. Hence Gaussian error analysis 
may be used. In the limit when only statistical errors are taken into account, the standard 
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Figure 4: The muon event rate for HK (1.8 Mt yr) with 10° angular smearing and 15% energy smearing 
plotted versus cos6'in taking 18 bins in the range -1.0 to -0.1, for 6 different energy bins in the range 4 to 10 
GeV, for both normal and inverted hierarchy. 



Gaussian definition of binned is: 



Xstat ^ ^ ^ ^ 

i=Enibins j=cos9nibins 



(23) 



Here, Nfj^ is tlie experimental and N^!* is theoretical number of events in the ij* bin. 

However, in addition to the statistical uncertainties, one also needs to take into account 
various theoretical and systematic uncertainties. In our analysis we include the uncertainties 
coming from 

• A flux normalization error of 20%, 

• A tilt factor [69] which takes into account the deviation of the atmospheric fluxes from 
a power law. 
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• A zenith angle dependence uncertainty of 5%, 

• An overall cross section uncertainty of 10%, 

• An overall systematic uncertainty of 5%. 



These uncertainties are included using the method of pulls described in [69-71] . This method 
allows us to take into account the various uncertainties in theoretical inputs and experimental 
systematics in a simple, straight-forward way. 

In this method, the uncertainty in fluxes and cross sections and the systematic uncertain- 
ties are taken into account by allowing these inputs to deviate from their standard values in 
the computation of N*!". Let the k**^ input deviate from its standard value by Uk ^k, where (Jk 
is its uncertainty. Then the value of with the changed inputs is given by 

npuU 

Nf = Nf (std) + 5^ (24) 
k=i 

where Ni|'(std) is the theoretical rate for bin ij, calculated with the standard values of the 
inputs and npull is the number of sources of uncertainty, which in our is case is 5. The ^k's 
are called the "pull" variables and they determine the number of a's by which the k**^ input 
deviates from its standard value. In Eq. [2ll Cy is the change in when the k**^ input is 
changed by cxk {i.e. by 1 standard deviation). The uncertainties in the inputs are not very 
large. Therefore, in Eq. [21] we only considered the changes in N*]^ which are linear in ^k- Thus 
we have a modified defined by 



1 2 

jex 



npull 



[ (std) + ZZf a - _ ^ 

X^(ek) = E ^ + E (25) 



ij k=i 



where the additional term is the penalty imposed for moving k*'^ input away from its 
standard value by cTk ^k- The with pulls, which includes the effects of all theoretical and 
systematic uncertainties, is obtained by minimizing ^^(^k), given in Eq. [23 with respect to 
all the pulls ^k^ 

xU = Mm^. U'(^k)] (26) 



4.2 Optimization of number of bins 



We do binning in the measured energy E^ and the cosine of the measured zenith angle (which 
is the measured polar angle of the incident neutrino) cos 6*111. For optimization purposes, we 
compute the with pull for various different choices of the number of energy and zenith 
angle bins. We found that the sensitivity for muon events improves with an increase in the 
number of bins, and is optimal for about 15 cos^m bins and 15 energy bins. For the electron 
events, the optimization occurs for a lower number of bins. This difference can be understood 
from the behaviour of Pee, P^e and P^^ in Figures [T] and [21 In the resonance energy range of 
4—6 GeV, oscillation probabilities involving i/g are relatively less sharply oscillating, whereas 
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the P^^ oscillates rapidly. Thus a finer binning in energy is needed for muon events to capture 
these oscillations, whereas a much coarser energy binning is enough to capture the change 
due to matter effect in electron events. However, because the energy resolution for a water 
Cerenkov detector gets poorer at higher energies in the GeV range, a very fine binning in 
energy for Em > 2 GeV is not realistic. Hence we consider a bin division into 8 Em bins in 
the range 2—10 GeV and 18 cos6'm bins in the range -1.0 to -0.1. The results given in the 
subsequent sections are with this binning. 



4.3 Marginalization over Neutrino Parameters 



In order to determine the optimum number of bins we have used the Xpuu from Eq. [26] where 
only the uncertainties in inputs such as fiuxes and cross sections are taken into account, holding 
the values of the oscillation parameters fixed in the calculation of both N^^ and N|j^. However, 
in general, the values of the mass-squared differences [Amgj^l and Am^j^ and the mixing angles 
(^12, 023 and 6*13 can vary over a range that reflects the uncertainty in our knowledge. Holding 
their values as flxed in computing both N^^ and is tantamount to assuming that they are 
known to inflnite precision, which is not realistic. To take into account the uncertainties in 
these parameters, we deflne the marginalized for hierarchy sensitivity as. 



A mm 



Min 



lAm^^l*™^ 



\Aml\ 



a(|Ami,|) 



+ 



sin^ 2^*™<= 



sin^ 26^ 



23 



a (sin^ 2^23) 



sin^ 261^" - sin^ 26 



13 



a (sin^ 2^13) 



(27) 



X^(^k) in the above equation, is computed according to the deflnition given in Eq. 



We use the following procedure for our analysis. 



• We simulate the number of events in 8 bins in the measured energy Em and 18 bins in the 
measured zenith angle cos 6^ for a set of "true values" for the six neutrino parameters - 
lAmg^l, 623, 0i3, Am2i, 612 and Sqp and for a "true hierarchy". This is our "experimental 
data" - N^^. 

• In the case of Am^i, O12, jAmgj^l and 6*23, the true values are the current best-flt values. 
The true value of ^cp is assumed to be zero in our analysis 0. 

• Matter effects, which are crucial for making a distinction between the hierarchies, are 
proportional to sin^ 26*13. At present there is only an upper limit on sin^ 26*13 (< 0.15). 
Therefore, we compute the for various different "true" input values of this parameter 
in its permissible range. 

^^The oscillation probabilities have only a weak dependence on Sqp for values of sin^ 26*13 which will be 
measurable in the forthcoming reactor neutrino experiments (> 0.05). In the next section, we will explicitly 
show that for both water Cerenkov and magnetized iron detectors, has only a weak dependence on Sqp- 
Therefore we set i5cp = 0. 
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• In order to test at what statistical significance the "wrong hierarchy" can be disfavoured, 
we calculate the theoretical expectation in each bin - N*^ assuming the "wrong hierar- 
chy" . 

• During this calculation of theoretical event rates we fix the solar parameters Am2]^ and 
612 at their best-fit values. Since the solar parameters have only marginal effect on the 
probabilities for the energies and pathlengths relevant for us, our results will not change 
significantly if these two parameters are allowed to vary in their currently allowed range. 
For the same reason we keep the CP phase 6cp fixed at its true value which we have 
taken as zero in this calculation. 

• However, we allow the parameters sin^26'i3, sin^26'23 and [Anigj^l to vary within the fol- 
lowing ranges : 

1. |Am|^| is allowed to vary in the range 2.35 x 10^^ — 2.65 x 10"^ eV^. 

2. sin^2^23 is varied between 0.95 and 1.0. However, P^o and P^^ in matter are 
dependent on sin^6'23. For sin^2^23 < 1 there exist two allowed values of 623 (the 
so called octant ambiguity). In our calculation we consider both values. Hence we 
consider a range 0.4 < sin^ 6*23 < 0.6. 

3. sin^ 26'i3(true) is varied from 0.0 to 0.15. The current 3a bound is sin^ 26*13 < 
0.15 [1]. 

• In computing Xmin' have added the priors for the neutrino parameters which puts a 
penalty for moving away from the true value. Shifting further from the true value of a 
parameter, would worsen the fit of the experiment which measured that parameter. By 
adding the priors we are effectively minimizing oi our data together with those of the 
experiments measuring the neutrino parameters. The results, obviously, depend on the 
choice of true parameter values. 

In the expression for Xmin' ^^i^ prior for the mixing angle 6*23 is given in terms of sin^ 26*23. 
This is valid because the quantity which will be measured in future z/^ disappearance 
experiments is sin^ 26*23 and the priors are added to take into account the fit to the data 
which made the measurements. 

In Eq. [23, 0" denotes la errors. We use 2% error for [Amg^l and sin^2^23! which can be 
achieved in future long baseline experiments [3]. For cr(sin^2^i3) we use 0.02 [24]. 

Our is thus marginalized over the three parameters sin^2^i3, sin^2^23 and [Amg^^l in 
order to determine Xmm which shows how different are the predictions of the the "wrong 
hierarchy" from those of the "true hierarchy". "Wrong hierarchy" is then taken to be ruled 
out at pa if Xmm — allowed values of 6*13, 6*23 and jAmg^^j. 

In our calculations, we took the density profile of the earth to be the PREM profile. 
There are, of course, some uncertainties in the values of the densities given in this profile. We 
checked that a 10% change in the density leads to a negligible change (less than 5% change) 
in the minimum x^. Therefore, we have not taken the uncertainties in the density profile into 
account explicitly. 
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Figure 5: Values of fixed parameter versus angular resolution (left panel) and energy resolution (right 
panel) for HK (1.8 Mt yr). Plots are given separately for xfi+p. ^'^d Xe+s- ^he left panel, an energy 

resolution of 15% is assumed, whereas for the right panel, an angular resolution of 10° is assumed. No 
marginalization over neutrino parameters is done. 



5 Results 



The ability of an atmospheric neutrino detector to resolve the hierarchy depends on various 
factors. We study the effect of these different factors by computing the ^ function of 

(a) energy and angular resolution of the detector and 

(b) sin^2^i3, which controls the matter effects. 

In studying (a) the same values of the neutrino parameters, | Amg^^l, sin^ 2^23 and sin^ 2^13, are 
used in computing both Nf^ and Njj*^. In studying (b) we marginalize over these parameters. 

For HK, the overall sensitivity is taken to be the sum of values for the muon and 
electron event rates and hence, the total = xfi+p. + Xe+e- 

For the I NO analysis, the fi~ and /i"*" event rate are separately used to compute the 
sensitivity to the hierarchy. The overall sensitivity is taken to be the sum of values for the 
/i~ and /i"*" event rates. Hence, for INO, the total = + 

5.1 Megaton water Cerenkov detectors 

We first discuss our results for megaton water Cerenkov detectors. 

The left and right panels of Figure [5] give the values of Xpuu from Eq. [26] for muon events 
ixfi+p) aiid for electron events (Xe+o) a function of angular and energy resolution respec- 
tively. In this figure all neutrino parameters are kept fixed in both N*'^ and N®^. Since the 
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Figure 6: Values of fixed parameter versus the input (true) value of sin^ 26*13 for HK (1.8 Mt yr), assuming 
NH to be the true hierarchy. Left panel shows xfi+p, ^-nd right panel shows Xc+s- energy resolution of 
15% and an angular resolution of 10° are assumed. No marginalization over neutrino parameters is done. 



number of muon events is expected to be more than that of the electron events, naively one 
would expect better hierarchy discrimination in the muon channel. We find that this indeed is 
the case when the energy and angular resolutions are very good. But with worsening energy 
and angular resolutions, the hierarchy resolving power of electron events becomes comparable 
to that of muon events. This is primarily because the contribution to the muon events comes 
from the probability P^^ and Pe^^ whereas for electron events it is from P^e and Pec- As is 
seen from the Figures [1] and Ej the matter effects in P^^ are generally smaller than those in 
P^e and Pee- Also, the matter induced change in P^e and Pee is consistently of the same sign 
over all the relevant ranges in energy and pathlength, while the matter induced change in P^^ 
is oscillatory for longer baselines. Worsening resolutions thus have a stronger effect on muon 
events, and, over-riding their statistical advantage, lead to xfi+p. < Xe+e- 

Figure [6] and Figure [7] give the values of the fixed parameter xfi+p, and xl+c functions 
of the true value of sin^26'i3. In Figure [6], NH is assumed to be the true hierarchy, whereas 
in Figure U\ IH is assumed to be the true hierarchy. The values in the two cases are quite 
close, as one would expect. Since 6*13 drives the matter effects, it is no surprise that the 
is larger for larger values of 6*13. It is interesting to examine the limit where 6*13 goes to zero 
for electron and muon events. For the former, one needs to recall that the relevant analytical 
expressions for Pee and Pe^^ in matter are those which are exact in Amg^/Amg^^ and accurate 
to first order in sin^ 2^13 [47-50] . The leading order term in this expansion is devoid of any 
hierarchy sensitivity, while the term to first order in sin^ 2^13 is zero. Thus we expect to 
go to zero in this limit for electron events, and this is borne out by the right panels of both 
Figure [6] and Figure [71 However, we see from the left panels of these figures that at 6*13 = 
the muon events have non-zero x^- The muon disappearance probability in case of 9is = 
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Figure 7: Values of fixed parameter versus the input (true) value of sin^ 26*13 for HK (1.8 Mt yr), assuming 
IH to be the true hierarchy. Left panel shows xfi+fi ^J^id right panel shows Xc+s- energy resolution of 15% 
and an angular resolution of 10° are assumed. No marginalization over neutrino parameters is done. 



can be expressed as, 



l-sin2 2^23 sin2|l.27(l-ac 



^12, 



A 2 L 

Ami - 

31 E 



(2^ 



Here, Ci2 denotes cos 6*12 and a is a dimensionless parameter given by a = Am2^/| Anig^^l. This 
gives us 



MM 



P^^, = sin^ 2^ 
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Thus there is a hierarchy sensitivity due to the term a = Am2i/|Am|]^| even for ^13 = 0. 



(29) 
This 



is true only if lAmgj^l is known very precisely, permitting the use of the same fixed value of 
lAmg^l for normal and inverted hierarchy. For current uncertainties in [Amg^^l, marginalization 
over this parameter leads to a wash out of this sensitivity (as will be shown later). This 
happens when the values of [Amg^^l in the computation of NH and IH probabilities differ by 
2Am|]^cf2 [72,73]. Hence we would have a non-zero hierarchy sensitivity for ^13 = only if 
the error in the determination of [Amg^l is less than the magnitude of Amg^^. 

Another interesting feature visible in Figure El and Figure [7] is that for sin^ 2^13 > 0.1, 
the Xe+e flattens out. As emphasized in Section 3, the maximum difference between events 
in the case of NH and those in the case of IH occurs for energies in the resonance region. 
At resonance, the matter dependent mixing angle ^ 7r/2, but the matter dependent mass- 
squared difference (Am|^)™ takes its minimum value of Am|j^ sin 2^13. For intermediate values 
of ^13, {i.e. those that are not tiny but still significantly less than 0.1) this mass-squared 
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Figure 8: Values of fixed parameter versus the input (true) value of 6cp for HK (1.8 Mt yr). Left panel 
shows xfi+fi s-J^id right panel shows Xc+c- -'^^ energy resolution of 15% and an angular resolution of 10° are 
assumed. NH is assumed to be true hierarchy. No marginalization over neutrino parameters is done. 

difference is too small for the oscillating term in P™ and P™ to come close to unity, even for 
the largest pathlengths. But for sin^ 2^13 > 0.1, the value of Am|]^ sin 2^13 is large enough such 
that (a) for a large range of pathlengths, sin^(1.27Am|^ sin26'i3L/£') ^ 1, (b) P™ is close to 
its maximum value, and (c) P™ is close to its minimum value. Thus, the difference between 
electron events for NH and those for IH becomes essentially independent of ^13 above a certain 
limit. 

Figure M gives the values of fixed parameter xfi+p. and Xe+e ^ function of the value of 
the CP phase Sqp for muon and electron events. This figure shows that the dependence of 
on 6cp is mild. This occurs because the terms containing Sqp in oscillation probabilities are 
also proportional to Am^^, whose effect on these probabilities is small. Therefore the exact 
value of 6cp has negligible effect on the hierarchy distinguishing ability of megaton water 
Cerenkov detectors. 

Finally, we present results with marginalization over the neutrino oscillation parameters 
|Am|;^|, sin^ 2^23 and sin^ 26*13. We add the of the muon events {xfi+p.) and that of the 
electron events (Xe+e) and compute the set of where all the inputs in Nj-*^ have been varied 
over their allowed ranges. The minimum of this set, called Xmin (-^l- 123), is the quantity 
characterizing the capability of HK to distinguish the two hierarchies. As explained earlier, 
this capability is a function of the true value of 6*13. In Figure [9] we plot the values of the 
marginalized xfi+fi + Xc+c ^^e true value of sin^ 26*13. The x^ curve is flattened for sin^ 2^13 > 
0.1 because of the contribution of Xe+g' which, as discussed earlier, flattens out for higher 
values of ^13. We find the x^ to be > 4 for sin^ 2^13 > 0.05. Thus if D-CHOOZ finds a non- 
zero value for sin^2^i3, then the combination of muon and electron events at megaton sized 
water Cerenkov detectors can make a distinction between the two hierarchies at 95% CL over 
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Figure 9: Values of marginalized versus the input (true) value of sin^ 20i3 for HK (1.8 Mt yr). Shown is 
the sum of xfi+fi ^'id Xo+e- energy resolution of 15% and an angular resolution of 10° are assumed. 

a ~ 3 year exposure period. Note that, for sin^ 26*13 = 0.05, the marginahzed is about half 
of the fixed parameter x^- 

We have not included the backgrounds in our calculations. They will be the same in both 
"experimental" and "theoretical" spectra and cancel out in the numerator of Xstat^ can be 
seen from Eq. [231 They need to be included in the denominator, which is the sum of "signal" 
and background events. Thus the denominator will be slightly larger than the computed 
number N^^ and Xstat ^i^l be correspondingly lower. Thus the backgrounds can be adequately 
taken care of if we divide Xstat of muon events by 1.003 and that of electron events by 1.17. 
We have not included these factors in our calculations but their inclusion makes a very small 
change in overall x^- 

5.2 Magnetized Iron detectors 

As stated previously, the iron detector which we use as a prototype (based on the I NO design) 
has a mass of 100 kT and is capable of separating the z/^ and events, but is insensitive 
to any electron events. We assume an exposure time of 10 years and first compute the fixed 
parameter Xp^jj from Eq. [^Hl and later the marginalized Xmin i^'i- both cases, we 

compute xfi and xfi separately and add them. The input values of neutrino parameters are 
the same ones used in the case of water Cerenkov detectors. 

In Figure [10], we plot the fixed parameter xfi + as a function of angular resolution 
and energy resolution. From the left panel, we observe that the increases sharply (by 
a factor of 3 or more) if the angular resolution is improved from 10° to 5°. This occurs 
because the maxima and the minima of P^^ are narrower in width and exhibit a more rapid 
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Figure 10: Values of fixed parameter versus angular resolution (left panel) and energy resolution (right 
panel) for I NO (1 Mt yr). Shown is the sum of xfi and Xp.- For the left panel, an energy resolution of 15% 
is assumed, whereas for the right panel, an angular resolution of 10° is assumed. No marginalization over 
neutrino parameters is done. 

variation (compared to Pee) as the baseline changes, as can be observed in Figures [T] and [21 
The improvement in due to the improvement in the energy resolution is more modest, as 
the right panel of Figure [TO] shows. Thus it is imperative for such detectors to improve their 
angular resolution to the best of their ability [38] . 

In Figure [TTl we plot the fixed parameter xfi + as a function of the input (true) value 
of sin^ 2^13. The left panel assumes NH as the true hierarchy while the right panel assumes IH 
as the true hierarchy. As in the case of muon events in water Cerenkov detectors, increases 
for increasing values of sin^ 2^13. And there is a small non-zero for ^13 = because of the 
effect of the non-zero value of Am2]^ discussed earlier. 

Figure [12] demonstrates the variation of xfi + Xp. with 6cp keeping the other parameters 
fixed in both NH and IH. As in the case of water Cerenkov detectors, the in this case is 
also insensitive of 6cp- 

Finally, we marginalize over the neutrino oscillation parameters lAmgj^l, sin^ 26'i3 and 
sin^ 2^23 and present the marginalized xfi + Xp, foi' the magnetized iron detector in Figure [13] as 
a function of the input value of sin^ 2^13. Here again, we find that resolution of mass hierarchy 
at > 95% C.L. is possible for values of sin^ 26'i3 > 0.05. The demonstrates a steep rise 
with an increase in ^13 which is due to the charge identification capability of magnetized iron 
calorimeter detectors. For this type of detectors we add the from muon events and that 
from anti-muon events, each of which is a sensitive function of ^13, and therefore the total 
has a large 6*13 dependence. Note that the x^ for ^13 = is no longer non-zero due to the 
marginalization over |Am|^|. 
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Figure 11: Values of fixed parameter versus the input (true) value of sin^ 20i3 for I NO (1 Mt yr), assuming 
NH (left panel) and IH (right panel) to be the true hierarchy. Shown is the sum of xfi and Xp- An energy 
resolution of 15% and an angular resolution of 10° are assumed. No marginalization over neutrino parameters 
is done. 



6 Summary 



In this paper, we have studied the hierarchy resolving power of (a) megaton sized water 
Cerenkov detectors (prototype: HK) and (b) magnetized iron detectors (prototype: INO). 

The first class of detectors has two important advantages: (a) a very large size leading to 
high statistics and (b) the ability to detect both muon and electron events. However, these 
detectors are insensitive to the charge of the lepton and therefore one needs to sum over the 
lepton and antilepton events. The matter effect affects the lepton events for NH and antilepton 
events for IH, and in the summed event rate these effects can cancel each other. In actual 
practice however the cancellation is only partial since the neutrino cross section for producing 
leptons is about 2.5—3 times higher than the corresponding anti-neutrino cross section. This 
leads to hierarchy sensitivity in these detectors, and despite the above disadvantages, the sheer 
weight of statistics can lead to significant differences in the signal for the two hierarchies. 

For the binned analysis of atmospheric neutrino data it is important to include the 
effect of energy and angular smearing. This effect is more significant for muon events than for 
electron events. This is because the matter effects in P^^ are generally smaller than those in 
Pee and P^e over the energy and baseline ranges under consideration here. Also, P^^ exhibits 
an oscillatory behaviour with energy at the longer baseline values (~ 10000 km) which tends 
to get washed out with smearing. Pee and P^e, on the other hand, have a significant matter 
effect over a broad range of energies and pathlengths, which is less affected by smearing. Thus 
the electron events gain in significance, and the cumulative effect from a number of bins adds 
up to create a large difference between the hierarchies, even though the electron neutrino flux 
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Figure 12: Values of fixed parameter versus the input (true) value of 6cp for I NO (1 Mt yr). Shown is 
the sum of xfi and xfi- ^.n energy resolution of 15% and an angular resolution of 10° are assumed. NH is 
assumed to be true hierarchy. No marginalization over neutrino parameters is done. 



sin^ 2^13 


Xl+fl + Xl+e (HK) 




0.0 


0.0 


0.0 


0.04 


3.6 


4.5 


0.10 


5.9 


9.6 


0.15 


7.1 


16.9 



Table 4: Values of total marginahzed x^ with pull and priors, for HK (1.8 Mt yr) and INO (1 Mt yr). 



is only about half of the muon neutrino flux. Together, the differences induced in /i-like and 
e-like events lead to a 2a signal for neutrino mass hierarchy for sin^ 2^13 ^ 0.04 for a moderate 
exposure time of 3.3 years. For larger exposure times, it is possible to determine hierarchy at 
a higher statistical significance. 

Magnetized iron calorimeter detectors, on the other hand, are sensitive only to muons. But 
the magnetic field endows them with charge identification capability. Therefore one can collect 
the muon and the anti-muon events separately and compute the x^ for each type of events and 
then add these. Thus the hierarchy sensitivity is considerably enhanced as compared to water 
Cerenkov detectors, and similar statistical significance can be achieved with smaller statistics. 

In Table H] we compare the sensitivity of both type of detectors for different values 
of sin^2^i3. We find that an exposure of about 2 Mt yr for water Cerenkov and 1 Mt yr for 
magnetized iron detectors can resolve the matter hierarchy at 95% C.L. or better, provided 
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Figure 13: Values of marginalized versus the input (true) value of sin^ 29i3 for I NO (1 Mt yr). Shown is 
the sum of xfi and Xp.- An energy resolution of 15% and an angular resolution of 10° are assumed. 

sin^ 26*13 > 0.05. This table and Figures [9l and [T3] summarize our main results, where Xmin is 
plotted versus the input value of sin^ 26*13 . This Xmin incorporates theoretical, statistical and 
systematic errors, along with smearing over neutrino energy and direction and also marginal- 
ization over the allowed ranges of neutrino parameters. These figures as well as the table show 
that the variation of with increasing sin^ 2^13 for water Cerenkov detectors is significantly 
flatter as compared to that for magnetized iron calorimeter detectors. This is due to (a) The 
contribution of xl+e the for HK, which flattens out for higher values of 6*13; and (b) xfi+fi 
is a relatively less sensitive function of ^13 since the addition of muon and anti-muon events 
partially cancels the 6*13 dependence in the difference in NH and IH events (Eq. [T71) . On the 
other hand, for magnetized iron calorimeter detectors is the sum of the from muon 
events and that from anti-muon events and the 6*13 sensitivity adds constructively in this case. 

The x^ foi' muon events exhibits a higher sensitivity to improved angular and energy 
resolution than do electron events. Hence, there is a dramatic improvement in the hierar- 
chy discrimination capability of magnetized iron detectors with improved angular resolution. 
The results quoted in the previous paragraph assume a modest energy resolution of 15% and 
angular resolution of 10°. Increasing the the angular resolution from 10° to 5° dramatically 
increases the x^ by a factor of 3. Improvement in the energy resolution for a magnetized 
detector leads to a more modest improvement of the x^- Thus it is imperative for these detec- 
tors to have good angular resolution if they are to resolve the mass hierarchy at a statistically 
significant level. 
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A Calculation of the normalization constant 



In this appendix, we give the procedure for computing the Gaussian resolution function for 
angular smearing. 

The flux distribution in terms of the zenith direction is given as (d$/dcos^t)- Thus the 
flux per solid angle is given by (l/27r)(d$/dcos6't). The number of events per unit solid angle 
are given by 



dN, 



1 

2^ 



d<I) 



d cos Of 



d$ 



d cos Of 



(Al) 



Even though the left hand side says Qt, this quantity is independent of (pf This plays a 
key role in later calculation. In the above expression, the disappearance and appearance 
probabilities are functions of neutrino travel distance and hence of 0^. In addition, they and 
the cross section are also functions of neutrino energy, which we will not explicitly consider 
in this discussion. 

Now consider a unit sphere with two nearby points whose coordinates are {Of,(f)t) and 
{Om, 0m)- The distance between them is given by 



ds^ 



0. 



sin^ Of 



(A2) 



Let be the uncertainty in the determination of the neutrino direction, which is given by 
the point (^t, 4>t)- Then the smearing factor for the solid angle is 



R(fit,^^m) = Nexp 



Or, 



sin^ Of 



2 (A^)^ 



(A3) 



Here, the normalization factor N = 1/A(^t), where A(^t) is the area of the Gaussian for each 
value of Of. Since the Gaussians at the edges are truncated, this causes a fall in the area of the 
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side bins if the normalization factor used is \f7^o (the area of a full Gaussian). We require 
the total event number to be preserved when the range of cos^m is the full range (-1 to +1), 
since any value of cos Q outside this range is unphysical and hence no events should come in 
or be thrown out. So, the normalization has to be by the actual area of each Gaussian. 



For each value of ^t, the Gaussian area is given by 



A(^t) 



d6'm sin6'in exp 



U2 + sin2^^ 



2 (A^)^ 



(A4) 



On computing the </)ni integral, using the same method described below in Eq. IA8HA10[ we get 



dcos6'in exp 



\2n 



27r 
sin 



(A5) 



Combining the smearing factor from Eq. [2T] with the distribution in true angles coming from 
Eq. lAll and integrating over 0t and Qt we get the distribution in measured angles 



dN, 
dfi„ 



dN, 
dfit. 



(A6) 



where df2t = d^t sin6'td0f In the above equation, 0t appears only in the smearing factor and 
it appears only as (0t — 0m)^- We can define this as 0^ so that the integrand is completely 
independent of 0t and that integration is trivial. Thus the distribution in measured angles is 
given by 

dN r /dN \ 

" (2^) y (37^ ) R(a,^^m) sin^td^t (AT) 



dl^t 



Now we do the integral over so that we have the distribution over cos 6*111 on LHS and 
cos^t on RHS. On RHS, only the smearing factor R(f2t,fim) depends on </)m. Thus doing the 
integral we get 



dN, 

dfirr 



d0r 



27r 



TT ^0 



dN^ 
dl^t 



N(^t) exp 



(^t-^m)' + sin2et 



2 (A^)2 



sin d6't d0n 



(A8) 



Changing variable to (pm = (pm — 4>t, we get 
dN„ , , ^ r r dN„ 

n Jo 



da 



27r 



dfit 



N(^t) exp 



2 (Ae)2 



sin6't dOf 



(A9) 



In doing the 0jn (0m) integration, we extend the limits of integration from ±7r to ±00. This 
leads to neghgible error because the Gaussian falls off very steeply. After the integration, we 

get 



dN r dN 

= 2n ^ m) exp 
dcosfc'm Jo dilt 



sin 9f d9f V27i A9 



sin^t 



(AlO) 
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We can cancel one factor of sin6't on the RHS. On substituting for N(0t) from Eq. IA5I and 
dN^/df2t from Eq. lAlt we get the binned distribution in cos^m to be 



dN„ 



d^. 



d^t 
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